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MULTIPLE  INTEGRAL  EXPANSIONS  FCH  NONLINEAR  FILTERING 
S.  K.  Mitter  and  D.  Ocone 
Massachusetts  Institute  cf  Technology 
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1.  Introduction 

In  their  seminal  paper,  Fujisaki,  Kallianpur  and 
Kurdta  [ l]  shoved  hov  the  best  least  squares  estimate 
of  a  signal  contained  in  additive  white  noise  can  be 
represented  as  a  stochastic  integral  with  respect  to 
innovation  process,  the  integral  being  adapted  to  the 
observation  process.  The  difficulty  with  this  repre¬ 
sentation  is  that  in  general  this  estimate  is  not 
useful  for  computing  the  estimate  since  the  innovations 
process  depends  on  the  estimate  of  the  signal  itself. 

In  this  paper  we  discuss  representation  of  the  estimate 
directly  in  terms  of  the  observation  process.  In 
doing  so,  we  derive  new  results  on  multiple  integral 
expansions  for  square-integrable  functionals  of  the 
observation  process  and  show  the  connection  of  this 
work  to  the  theory  of  contraction  operators  on  Fock 
space.  This  letter  development  is  due  to  Nelson  and 
Segal. 

We  also  present  several  applications  of  these 
results  to  determining  sub-optimal  filters. 

2.  Multiple  Integrals  and  Filtering 

In  this  section,  we  shall  discuss  applications 
of  multiple  integral  expansions  to  the  general  fil¬ 
tering  problem.  We  will  consider  the  'canonical* 
scalar  filtering  model: 

yt mC  h(xs)ds  +  vt  (1) 

under  the  assumptions 

a)  x^and  w^  are  independent  processes 

b)  for  some  T>0  E/^h^(xg )ds<"  (2) 

c)  »t  is  a  standard  Brownian  motion 

If  f. (x,  ,)=f. (x  ,s<t)  is  a  causal  functional  of  the 
t  \’ )  t  s  — 

signal  xt  and  F^=a{yg | 0<s<t}=  sub-o-algebra  generated 
by  yg , 0<s<t ,  then  we  are  interested  in  calculating  the 

optimal  least  squares  estimate  of  ft(x^  )) 

Etft(x^ . j ) lF^>  for  t<T. 

Definition  1  yf  defined  in  (l)  and  (2)  is  called  an 
observation  semi-rr.artir--ale ■ 

Throughout,  let  (il,F,P)  denote  the  underlying  proba¬ 
bility  space. 

Now  E{f  (x)|r,y}eL(n,ry,P)  (={1^  measurable  rv's}) 

Xu  w  X 

by  the  definition  of  conditional  expectation,  and, 
therefore,  any  method  that  represents  elements  of 

Lffi.F^.P)  in  a  simple  and  consistent  way,  say  by 
expansion  in  terms  of  a  simple  class  of  functionals  of 
yj . ^  can  be  applied  to  the  optimal  estimate.  In 

this  work,  we  have  adopted  multiple  integrals  of 
the  form  /t..^Sr-1k(t,s1,.  .sr)dy(sr).  .dyCs^ 

as  the  basic  objects  of  expansion.  First,  y^  is  a 
stochastic  translation  of  Brownian  motion  and 
through  a  change  of  measure,  much  Brownian  theory  can 
be  carried  over.  Secondly,  iterated  integrals  provide 
the  natural  concept  of  a  polynomial  in  the  y  process 
and  thus  they  give  a  framework  for  considering  best 
quadratic,  cubic,  etc.  sutoptimal  estimation  pro¬ 
cedures.  Finally,  when  the  kernel  k  of 

VM8r-lMy.  "dys*  £er arable >  a  construction  of 

Brockett(2]  real! tea  recursively  as  the  solution 


to  a  stochastic  differential  equation. 

Accordingly,  after  developing  some  theory  of  multi¬ 
ple  integral  expansions  ve  show  how  E{ft(x^ . | F^ }  can 

he  represented  as  a  ratio  of  multiple  integral 
expansions.  The  chief  theoretical  result  about  multi¬ 
ple  integrals,  the  multiplication  formula  of  theorem  2, 
is  then  used  in  conjunction  with  this  representation 
to  derive  equations  for  the  best  suboptimal  estimate 
of  any  order.  The  Kalman  filter  is  derived  and  the 
quadratic  filter  discussed  in  detail  as  examples. 

Multiple  Integrals.  Inwhat  follows,  let  (b(t),F  ) 

denote  a  standard  Brownian  motion  v.r.t.  increasing 
family  of  sub-o-algebras  F.  .  We  assume  familiarity 

“  +  0 

with  the  stochastic  integral  /  <(>gdb(s),  where$g(u)  is 
a  measurable  process  adapted  to 

Definition  2  Let  fcL2(  l O^T ]n )  =  CfeL2(  (0,Tjn)  |f  symmetric}. 

1^°^  (f),  the  nth  order  multiple  (or  iterated)  integral 
up  to  t<T  of  f,  is  defined  recursively  by 

I{t,)(f)=^I^-1)f(s,..)db(s)  ^  (3) 

In  (3),  f(s,..)  is  the  function  of  L2(lO,Tjn_1)  formed 
by  holding  the  first  element  of  f  fixed  at  s.  Strictly 
speaking,  for  (3)  to  make  sense  it  must  be  shown  that 

j(n  has  a  measurable  version,  but  this  can 

easily  be  done  by  approximating  f  with  separable 

functions.  Let  (f  .g^A/^fCs  ,..,s  )g(s ,..,s  Jds  -ds, 
oo  ±2  n  j.  n  n  1 
denote  the  inner  product  of  L  ([0,T]n).  By  applying 
standard  facts  about  stochastic  integrals,  the  following 
basic  properties  of  the  multiple  integral  are  derived: 

for  any  n  and  m,  t<T,  and  f£L2( (0,T)n) ,  gcL2([0,T]n) 

a)  E{l*n)(f)}=0  (U) 

.  »  „  T(n)r  ,  (m),  ,  rO  if  mHn 

b)  E  xt  CfUt  If  n=n 

Note  also  that  l£n^(f)  depends  only  on  the  values  of 
f(slt..,sn)  for  s1>Sg>->sn.  (3)  adopts  the  useful 

convention  of  allowing  f  to  be  defined  in  all  of 

[0,T]n  by  a  symmetric  extension.  ** 

Multiple  integrals  are  useful  in  constructing 
Wiener's  homogeneous  chaos  expansion,  which  as  an 
example  cf  the  general  theory  presented  later,  decomposes 

L  (F^)  into  a  direct  sum  of  Hilbert  space  tensor  ' 

products.  Indeed  if  H^R,  Hn=U^(f )  |f£L2(  [0,T]n)J  n>l 

a  simple  application  of  (U)  a)  and  b)  demonstrates  that 
Hn  is  a  Hilbert  space  for  every  n  and  that  H^Hm  for 

n/m  [where  _[_  is  defined  in  the  sense  of  the  inner 
product  (x,y)=Exy).  In  fact  we  have  more: 

Theorem  1  (Ito-Wiener) 


l2(f£)=ho«  H2e 


That  is,  for  4’£L2(F^)  kernels  (k  }  exist  such 
<»  in)  v  n  n*=0 

^oVt'knl  ( 


t  Ctf 


Proof.  See  Ito  [j  and  Kallianpur  [4. 

Theorem  1  suggests  the  following  natural  question. 

Suppose  fcL2([0,T]n)  and  geL2(  [  0,T]n) .  Is  it  then  true^J^ 
that  l[n)(f)l*n\g)eL2(F*)  for  t<T,  and  if  to,  what  . 
are  the  kernels  {k^},  as  in  (5),  such  that  l”(f)I*(g)  * 
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*k +£.  _l'  '(k. )?  Our  answer,  vhich  will  become  a 
o  1*1  t  1 

principal  tool  of  investigation,  requires  some 
preliminary  notation. 

Definition  3  i)  P  will  denote  the  projection  of 
L2([0,T]n)  onto  L2([CVr]n) 

{Pnh)(sl"Sn)s7:^es(n)h(37I(l)’"ST,(n)) 
vhere  Sn=permutation  group  on  n  letters. 

ii)  Let  0<k<nin(n,n)  fel2( [0,T]n) ,  geL2([0,T]“) 
(ffiklt)g)(s1,..,sn+c_2k)  (g) 

^k !  ^"£tf  (rl*“  »rk  ’  S1  *"  sn.k  rl"rk  *  sn-k+l"sm+n-2k  5drk’'drl 

(f8kCt)g)(s1..sn+n  2k)=(?^+n_^[fa  (t)g] )(s1-sra+n_2kj  (7) 

To  illustrate,  if  n>m=k,  then,  “sing  the  symmetry  of 
f  knd  8 , ( f9m ( t ) g K Sj- sn_k )I  ^^tf(r1,..rm,s1..3ii_m) 

xg(r,..r  )dr  ..dr  ,  It  is  useful  to  think  of  the  func-' 
i  m  m  i. 

tions  f  and  g  as  tensors,  for,  in  fact  Ll[0,T]n) 

=L2[0,T)fl..2L2([0,T) )  (n  times).  Therefore,  as  inspec¬ 
tion  of  (6)  and  (7)  suggests,  0r(t)  may  be  interpreted 
as  a  k-fold  symmetrized  tensor  contraction. 

Theorem  2.  Let  feL2([0,T]n),  geL2(.[0,T)m) .  Then 
I(n)(f)I(m|?)£L?(Fb)  for  ^  and> 

Before  sketching  a  proof,  let  us  first  demonstrate 
that  the  l.h.s.  of  (31  is  vell-defined, 

Lemma  1.  Let  feL2([0,T]n) ,  GeL2(l0,T]m).  For  t<T 
f9k(t)gcL2([0,T]n+n'2k). 

In  fact  lire  )« lpa+n_2k<c  Hr  if  lie  |g.  Cftl 

vhere  C  depends  on  a,  n  and  k. 

Proof,  Let  |Sn|  =  cardinality  (S^I  and  J=m+n-2k. 

Using  Cauchy-Schvarz  repeatedly: 

and 


l|fV  t>E(sir(i)*sir( j ) > I? ■ “fez  Wa,w(i>-4Mj > 

x;I^t7tf(Vsr-s1r(l)")e(rl"rr-sir(j))drr"drl]2 
Thus  l^t^lIjiljSpJ^lflglbt 

Proof  of  theorem  2*:  Only  a  sketch  will  be  given,  as 
details  are  involved  and  unrevealing.  First,  it 
suffices  to  treat  the  case  when  f  and  g  are  separable, 
since  ve  can  use  lemma  1  to  approximate  general 
f  andg  by  separable  functions.  This  makes  questions 
concerning  the  interchange  of  dt  and  db(t)  integrations 
easy  to  resolve.  The  case  n=l,  m=l  follows  directly 
by  applying  Ito’s  differentiation  rule.  Indeed,  Itp’s 
rid.c  yields  in  general 

I(n)(f)i(m)(g)  „  /tI(™>(6)I("-l)(1.(s>..))db(s)  +  (10) 

^n)(f)l'r-1)(g(s,..))d1b(s)+^Vn-l)(f(r.,..)) 

*I*n'1)(e(e..))ds 

Using  (10)  ve  can  implement  the  following  two  stage 
Induction  scheme  to  prove  the  theorem  for  all  m  and  n, 

•Apparently,  Japanese  workers  have  also  recently 
proved  theorem  2  by  means  of  functional  analytic- 
techniques  due  to  Hida  (Personal  communication  fromT.llida) 


a)  The  case  (m,n)  =  (k,l)  implies  the  case  (s,b)  =  6c*TJ) 

b)  The  cases  (n,n)=(k-l ,J ) ,  (k,J-l)  and  (k-l,J-l) 
imply  the  case  (K,j). 

Equation  (8),  the  multiplication  formula,  is  actu¬ 
ally  a  generalization  of  similar  looking  Hernite 
polynomial  identity 

where  h  (x)  e-x2^2(dn/dxn)ex2^2’  To  understand 


the  connection,  observe  that  the  polynomials  h_(x) 
provide  an  alternate  means  of  constructing  the 
decomposition  of  theorem  1.  In  fact,  if 
is  a  complete  orthonormal  basis  of  L2([0,T])  and 

Gn  5  Span$  !  \li\ ( J ) ( s )db( s ) )  I  V"+Pr=n •ii*"ir 

pairwise  unequal)  _ 

then  Ito  [  3  ]  has  shown  that  H  =G  (  denotes  closure). 

n  n 

(See  also Kallianpur  [  4  ]).  Thus  a  typical  element 

lfn^(f )eH  is  a  generalization  of  a  Hermite  polynomial, 
u  n 

The  slight  discrepency  between  the  factors  in  (11)  end 
(8)  arises  from  the  normalizations  involved  in  the 


definitions  of  h 


I*n*  and  fij^. 


(8)  has  consequences  that  relate  directly  to  the 
theory  of  contractions  on  sums  of  Hilbert  space  ter.sor 
products  presented  in  a  later  section.  The  point  is 
that  the  multiplication  formula  can  be  used  to  study 
the  integrability  of  kth  order  moments  of  the  integral 
(n) 

1^  ;(f),  and, indeed,  a  direct  application  of  (8)  using 
lemma  1  and  a  recursion  argument  yields: 

O  n 

Theorem  3  Let  n>l  and  feL  ([0,T]  ).  For  any  k>l, 
there  exists  M  ' >0,  independent  of  f,  such  that 


E(i;n,(f)]^X,kllf1lm  *  (12) 

How,  Segal  [  5  ]  has  previously  derived  (12)  by  tensor 

product  operator  arguments,  and,  in  addition,  proves 
there  exists  a  constant  c  such  that  M(m,k)  nay  be 

replaced  by  k2ckn.  His  argument  thus  connects  (12) 
to  a  deeper  general  theory. 

Theorem  3  has  an  interesting  corollary. 

Theorem  It  Let  {f  }  .  and  f  be  functions  in  L2(fC.Tin). 

■  1  ei  ra=i 

jfr  Hv*?"0  iff  E(4n)(fffi)-4n)(f)32  =  ajt 

iff  lin  E[4n)(fn)-4n)(f)]2k*°  for  any  or  all  k. 

Proof  By  (Ub)  E[ 4" } (1‘m)-4n ’ (f  ^  “  nl  ^m"f  l£. 

Using  (12)  completes  the  proof. 

In  the  applications,  ve  shall  want  to  discuss 
multiple  integrals  not  with  respect  to  Brownian  r.cticn, 
but  to  an  observation  semi -mart ingale  y. .  We  again 

fnl  ' 

denote  these  integrals  by  I^n,(f)  without  explicitly 
indicating  the  dependence  on  y^,  vhich  should  be 

clear  from  the  context  of  their  use.  The  simplest 
definition  of  such  an  integral  uses  a  result  stated  in 
theorem  namely,  under  condition  (2)  there  exists  .  \ 
a  measure  PQ  on  (0,F)  such  that  i)  yfc  is  Brovniar.  < 

on  (fi,F,Po),  and  ii)  P0«nd  P  are  mutually  absolutely^*, 

continuous. 

Definition  l«  For  fEL2([0,Tjn) 

4n,(f)^V(n’l)f(V8n)dy(8n,"d3r(8l)  *3^ 

is  a  r.va.s.  equal  to  the  Brownian  multiple  integral*-^ 
defined  on  (tl,  F,  Pq). 


2 


f 


?y  absolute  continuity,  I^n^(f)  is  a  veil-defined  r.v.  lj(s^-Sj)=E[h(Sj)-h(Sj)] 


on  (ft,  F,  P).  Also,  as  further  argument  vill  show, 
l|n^(f)  equals  the  iterated  integral  defined  directly 
on  (0,  F,  P)  in  the  manner  of  definition  2,  and  thus  the 
'natural'  interpretation  of  l£n^(f)  as  an  iteration 

is  preserved.  It  immediately  follows  from  definition  U 
that  the  multiplication  formula  holds  for  the  obser- 

dP  2 

vation  semi-martingale  case.  Likewise,  if  E  (or  )  <«> 

O  aro 

then  E[4n\f)]2k  =Eo§-o ( F )] 2k  < ( ) 2  /*  (  Eq  [  I  <  n  }  (  f )] Uk| 

shows  that  theorem  3  extends  as  well. 

Finally,  it  is  important  to  compute  the  mean  and 
variance  of  integrals  with  respect  to  y^. 

Lemma  2  Let  E(^ih2(xs)ds)n<'so.  Then  for  k<m  and 
feL2(iO,T]kl, 

11  ElIk(f)J2£Mk[>f1!^  (M.r-does  not  depend  onf) 

til  EllJCfl]  =^./S^l)f(s1..sr)E{h(xsU)..h(xsfe.}}dM.d£Cl) 

Proof  The  proof  proceeds  by  induction  on  the  order 
k,  and  the  induction  stops  at  k=n  because  of  the 

condition  E(/th2(x  )ds)n<".  Details  will  not  be 
o  s 

presented  for  lack,  of  space. 

Filter  expansions  and  applications.  We  will  now 
show  that  the  Kallianpur-Striebel  formula,  (13), 
for  the  optimal  estimate  can  be  developed  into  a 
representation  of  the  estimate  es  a  ratio  of  two  mult¬ 
iple  integral  expansions. ■  This  technique  bears  compar¬ 
ison  to  the  work  of  Eterao  [6  ],  who  derived  simular 

expressions  in  an  effort  to  approximate  the  conditional 
distribution  of  the  signal  given  the  observation 
process.  Here  we  focus  on  the  use  of  the  expansion 
to  derive  equations  for  suboptimal  filters. 

Recall  the  filtering  problem  l)-2).  Denote 
h(x(s))  by  h(s), 

f.(x(* ))  by  f(t),  and  ElfCtJlF^Jby  f(t),  and  define 


•  L(t)=exp[/th(s)dy(s)-l/2^th2(s)ds] . 

Also,  define  a  new  measure  P  on  (Q,F)  by 
H°=  exp  [-£th(s)iv(s)-l/2£th2(s)ds]. 

Theorem  3  Under  the  hypothese  of  (2l 

i)  P  is  a  probability  measure  and  P  and  P  . 

°  dP  ° 

are  mutually  absolutely  continuous  with  jpQ  =  L(T) , 

ii)  Under  P  y(.t)  is  a  Brownian  motion  independent 
of  x(t) . 

iii)  x(t)  has  the  same  distribution  under  P  as 
under  P 

iv)  (Kallianpur,  Striebel) 

E{f(t)|^}=E0{f(t)§o|^}/%{fo|^}  (13) 

=E0{f(t)L(t)|f^}/E0{L(t)|I^}. 

Proof  See  Wong  [7  ], 

Let  Lr_1(t)=^t../S^‘  ~^h( s3 )..h( sr  )L( sy  )dy ( s^J-dy ( s^) , 
Theorem  6  a)  Partial  expansion 

Suppose  E[^th2(o)ds)r<"  and  E[f2(sX£th^s)ds)r]«‘>, 
Then  ^).grt*)^lItJ)[fclHK0If(t)lT(t)|^1,(ll.) 

where  k^(t,  )nE[f(t )h(s1)-h(cj )]  and 


b)  Full  expansion.  If  E[exp[/  h  (s)ds])<“>  and 
Ef2(t)exp^"h2(s)ds<“,  then 

’  U5) 

where  k,  and  1,  are  as  above  and  the  infinite  series 
J  J  ^ 

both  converge  in  the  L  (P)  norm. 

Remarks  1.  The  kernels  k  and  1  depend  only  on  the 
J  J 

apriori  distribution  of  x(t). 

t  2 

2.  The  condition  E(exp[/  h  (s)ds]}«»  in  (6)  places 

o 

strong  restrictions  on  the  growth  of  the  moments  of 
T  2 

£  h  (s)ds.  Moreover 

E  (7!  )2  =  E  E  {exp[2/Th(s)dy(s)-/Th2(s)ds]!?^} 

O  GJTq  OOO  O  o 

.  =  E  exp(-/ih2(s)ds]E  {exp2/Th(s)dy(s) |F^> 

J  OO  OO  t 

=  Eoexp[^Th2(s)ds] 

U  y 

since  /  h(s)dy(s)  conditioned  on  F7  is  normal  with 
®  15  9  ^ 

variance  £  h  (s)ds. 

3.  Theorem  6  can  be  generalized  without  difficulty  to 
vector  valued  processes. 

Proof  of  theorem  6;  For  lack  of  space  ve  give  only  an 
heuristic  sketch.  The  principal  idea  comes  from 
observing  that,  by  using  Ito's  differentiation  rule 

dL(t)=h(t)L(t)dy(t).  In  other  symbols, 
L(t)=l+^h(s)L(s)dy(s)  (16) 

By  iterating  (l6): 

L(t)=l+^'h(s)dy(s)+^t/Sh(s)h(r)L(r)dy(r)dy(s). 

Continuing  such  iteration  ad  infinitum  we  derive  the 
formal  expression 

L(t)=l+z“  h(s^..h(Sj)dy(sj)-dy(s^) .  (IT) 

Now  substitute  (17)  into  the  term  EQ[L(t)fF^].  We  get: 
Eo[Lfe)|?^]=l+£j_1Eo{£t../SJ=lh(si)..h(Sj)dy(s1)..dy(s1)|?i 
=l+Ej=1^-^Sj  _1E0(h(  si )  -h(  s  j )  |  F^  }dy  (  Sj)  -dy(  sj 
_lE0{h(  )-h(  s  j  )  Idy  ( s , )  -4y  ( s3  ) 

(18) 

The  second  equality  uses  a  stochastic  'Fubini '  theorem 
found,  for  example,  in  Iiptser  and  Shiryaye.-  [s  );  for 
the  process  4>(s)  adapted  to  the  Brownian  motion 

(b  ,  F.  )  and  satisfying  E[/*V2(s)ds]<“ 

w  X.  O 

F.(^sdb(s)|F^)  =  ^TE[<>E|FgJdb(s). 

The  third  equality  follows  from  Theorem  5  ii)and  iii), 
and  the  fourth  equality  by  definition.  By  a  similar 
calculation, 

Eo(f(t)L(t)|F^}=l+E“=1l|j)(k5).  (19) 

Now  (18)  and  (19)  can  be  substituted  in 
f(t)  =Eo{f(t)L(t)|F^)/Eo{L(t)|F^} 

to  formally  derive  theorem  6 ,  b).  The  partial  expansion 
if  proved  by  carrying  out  the  iteration  procedure  of 
(16)  only  a  finite  number  of  times.  The  various 
hypotheses  in  theorem  C  merely  guarantee  that  the 
steps  in  each  calculation  are  valid. 
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3.  Applications 

The  explicit  formulas  (lit)  and  ( 15 )  can  be  applied 
to  the  design  of  subopt inal  filters  in  various  vays. 

For  example,  one  naive  approach  would  be  to  truncate 
the  numerator  and  denominator  of  the  ratio  at  finite 
orders  and  use  t'ne  result  as  an  approximate  filter. 

As  noted  in  the  remarks  after  theorem  6,  the  kernels 
of  the  expansions  do  not  involve  the  observations  y(-) 
and  so  can  be  computed  off  line.  Theoretically  then, 
it  is  possible  to  construct  the  truncated  filter. 

This  design,  however, is  difficult  to  analyze  and  assess; 
a  more  interesting  use  of  theorem  6  involves  finding 
estimates  that  are  multiple  integral  expansions  of 
finite  order. 

Definition  5.  a)  An  expression 

c(t)=c  (t) +  Er  .lfn)(d(t)) 
o  ^  n-1  t  n 

•with  cp(t)eL2(CO,T]n)  is  called  an  rth  order  expan¬ 
sion  of  y ( - ) . 

b)  An  rth  order  expression  a(t),  satisfying 
E[f(t)-a(t)]2<  E[f(t)-c(t)]2 


for  any  other  rth  order  c(t),  is  called  the  best  rth 
order  estimate  of  f(t). 


The  best  rth  estimate  will  be  denoted  by  f(t)  (with 

r  understood),  and  its  kernels  by  a  ,a, ,..a  . 

o  l  r 

Civen  an  order  r,  how  can  ve  find  f(t),  that  is 
how  can  we  determine  a  ,a.,..,a  ?  As  it  turns  out,  we 
can  apply  the  multiplication  foFmula  to  the  filter 
expansion  to  write  integral  equations  for  the  kernels 
an>  Begin  by  considering  the  product  f(t)Eo[L(t)  |F^] 

of  the  estimate  with  the  denominator  of  (13).  If 
E(^Th2(s)ds]2r<® 

then  the  expansion  of  EQ[L(t)|F^)  in  (lb)  applies,  and 

f(t)Eo(L(t)l^]=[ao(t)+^=1lJn)fen(t))Il+^iI(n)(ln) 

+  VVt)K}]=6oW+l=lIin!(8n^)>f(t)Eo{L2r(t)|^>.(2(« 

The  g  ,  n=l,..3r  are  calculated  from  a  (t)  and  1  (t) 
n  n  n 

by  use  of  the  multiplication  formula. 

Theorem  7.  Suppose  E(£Th2(s)ds]2r<"  ,  Ef2(t)<" 

and  Ef2(t)f/Th2(s)ds]2r<®. 
o 

Then  7(t)=a  (t)+Er  .ljn^(a  ( t ) )  is  the  best  rth  order 
o  n=l  t  n 

estimate  iff 


go(t)=Ef(t)  (21) 

e  (t,s  . .s  )=E{f(t)h(s  ). .h(s  )}=k  ,  l<n<r. 
n  in  i  n  n - 

Proof:  We  must  show  that 

E[f(t)-f(t)J2<E[f(t)-c(t)]2  (22) 

for  all  nth  grder  expansions  c(t)  iff  (21)  holds. 

Recall  that  f(t)  may  be  interpreted  as  the  projection 

of  f(t)  onto  E2(£l,  F^,  P).  Thus  the  projection 
theorem  implies 

E[f (f )-?(t) )2=E(f(t )-? (t) )2+E[f(t )-f(t) ]2 
♦  2E[f(t)-?(t))[f(t)-?(t)]=E(f(t)-?(t)]2+E(f(t)-/f(t)]2 


Applying  thi3  calculation  to  the  r.h.s.  of  (22)  also, 
(22)  holds  iff 


*[?(t)-f(t)]2<E[f(t)-e(t))2 


1< 


for  all  c(t).  But  according  to  lemma  2,  the  set  of  rth 
order  expansions  in  y( • )  is  a  Hilbert  space,  and  thus, 
applying  the  projection  theorem  again,  (23)  holds  iff 

E[f(t)-f(t)lc(t)=0  (2k) 


for  all  rth  order  expansions  c(t).  Now  substitute  the 
expression  (13)  for  f(t)  into  (2k): 

E  (f(t)L(t)F^)-f(t)E (L(t)|F^> 

E[f  (t  )-?(t)]c(t)=E(— - - - - - ° - -c(t) 

Eo(L(t)|^> 

^EEtH0  F^][Eo{f(t)L(t)|>^)-f(t)Eo(L(t)|F^]c(t) 
=Eo[Eo(f(t)L(t)|F^}-f(t)Eo(L(t)|F^})c(t)]  (25) 

The  second  equality  in  (25)  uses  the  identities 

E{£ci^Eo<§0^>rl 

=[e  (LU)!^}]-1, 

O  l 

which  are  easily  demonstrated.  How  under  P0«  y(  • )  is 

a  Brownian  motion  and  integrals  of  different  orders 
are  orthogonal.  Thus,  using  ( 20 )  and 

Eo{f(t)L(t)|r^}=Ef(t)+^=1l[n^(kn)+Eo(f(t)l.r(t)|l^} 


in  (25), 

E[ f (t )-f (t ) )c(t 5=2  [Ef (t )-g +1* if ° } (k  -g  ) ]c(t ) 
o  o  n-J.  t  n  n 

+Eo[f(t)c(t)Eo(D2r(t)|F^]  (26) 

+Eo[c(t)Eo(f(t)Lr(t)|^)) 


An  application  of  lemma  2  show  that  the  second  and 
third  terms  of  the  r.h.s  of  (26)  are  zero  for  all  c(t). 
Clearly,  the  first  tern  can  be  zero  for  all  nth  order 
c(t)  iff  k^=gE  for  0<p<r,  and  this  completes  the  proof. 

The  equations  ( 26)  are  actually  integral  equations 
for  the  kernels  aQ(t)  of  the  best  rth  order  estimate, 

since  the  gn(t),  CK n£r ,  are  found  from  ( t ) , 0<n<r , 

by  the  formula  (8)-To  illustrate,  if  r=l,  l^(s)=Eh(s)  and 

Ef(t)=go(t)=ao(t)+/ta1(t,u)Eh(u)dw 

Ef(t)h(s)=g1(t,s) 

=a  (t)Eh(s)+a  (t,s). 

O  1 

Solving  for  aQ(t) , 

a^(t  .sJ+^a^t.uJccvthCs)  ,h(u)  ]du=cov[f(t)  ,h(s)  J . 

This  is  the  familiar  Viener-Hcpf  equation  for  the  best 
linear  estimate.  In  the  best  quadratic  (r=2)  case,  the 
equations  become  core  complicated.  They  are,  assuming 
Eh(s)=0,  Ef(t)=0  fer  sinplcity, 

a2(t-  ui»  n2)Eh(u1)h(u2)dSgds1  (27a) 

a1(t,s)=Ef(t)h(s)-^ta1(t ,u)Eh(s)h(u)du 

Yo^1  e2^t,Ul’U2)c°v[h(s)  .hf^JhtugJJdUgdUj  (27b) 

a2(t ,s^,s2)=cov[ f( t ) ,h(sj) ,h(sg) ] 

-^a^(t  ,u)cov[h(  sx)  ,li(  s2)  ,h(u)  ]du 

-^t[a2(t,s1,u)Eh(s2h(u)+a2(t,S2,u)Eh(s1)h(u)]du 

°2^t  •u1,Ug)cov[h(s1)  ,h(s2),h(u1)h(u2)]du2du1(27c) 


(23) 


[In  (27),  cov  [X.  ,. .X  ]2E(X,-EX  ) . . . (X  -EX  ) . ] 
x  r  j  l  r  r 

(27)  shows  how  the  kernels  of  different  orders  are 
dependent  on  one  another.  Though  not  a  standard 
integral  equation,  (27)  may  he  reduced,  hy  using  the 
solution  of  a  related  linear  estimation  problem,  to  a 
single  integral  equation  for  a^.  For  fixed  t  this 
equation  is  of  Fredholm  type  for  a2(t,’,-)  and  can 
be  solved  by  standard  methods.  We  shall  not  go  into 
this  theory  here. 

The  multiplication  formula  can  also  be  used  to 
derive  the  Kalman  filter.  Consider  the  simple  case 

dx(t)=db(t)  x(0)=0 

dy (t ) =x (t ) dt+dw( t ) 


r(H)<n_i  in  r(H)<n.  For  f^..,^  in  H  we  define  the 

Wick  polynomial: 

:F(f.)...F(f  ): 

1  n 

to  be  the  orthogonal  orojection  of  F(f,  )...F(f  )  into 

1  n 

r(H)n.  In  the  special  case,  where  H  i3  one  dimensional 
and  hence  r(H)=L2(R,3(S)  ,(2Tr)_1^2e"x2/2dx>r(H)n  is  the 
one  dimensional  subspace  spanned  by  the  nth  Hermite 

polynomial  and  :xn:  is  the  nth.  Hermite  polynomial 
normalisation  so  that  the  leading  coefficient  is  1. 

We  have  the  formula 

[:F(f1)..F(fn):,:F(g1)..F(gn):] 


where  b(')  and  w( ' )  are  independent  Brownian  motions. 

Then  we  can  show  that  the  optimal  filter  is 

^(O^aft.sJdyCs) 
where  . 

a1(t,s)+^a1(t,u)Eb(u)b(s)=Enj(t)b(s).  (28) 

The  proof  is  simply  to  show  that  a(t,s)  can  be  chosen 
so  that 

?( t  )=^a(  t  ,s  )dy  ( s  )=Kf  (t  )+E“=1I^J,(kj) /l+£j=1I^J,(  ljX29) 

^ta(t,s)dy(s)[l+E“i=1I^j)(lJ)]=Ef(t)+E“=1I^)(kJ).(30) 

By  expanding  the  l.h.s  of  (29)  using  (8),  and 
equating  kernels  of  different  orders  we  derive  the 
infinite  set  of  equations. 

Ja(t,-)6o(t)lJ_1+a1(t,-)01(t)lJ+1=kJ.  (31) 

It  can  now  be  shown  that  if  (31)  is  satisfied  for 
J=l,  it  is  satified  for  all  jy,  a  result  following 
from  the  identity  for'  Gaussian  random  variables: 

lj(s^. .Sj)=Eb(s^) . .b(sj) 

=J'J=2c°v(b(s1)b(s2))E[b(s2).  ■b(sj_1)b(sj+1).  .b(sj)] 


_Mfir(l),g,^*tf  (n),en^  • 

where  the  sum  is  over  all  permutations  irof  l,...n. 

If  all  the  f's  and  g's  are  equal,  we  get 

[:F(f)n:,F(f)n]==-  i/2£(  : xn : ) 2e“x2/2dx=n ! .  (35) 

Let  H,  be  the  complexification  of  H  (and  let  H 
J-  n 

denote  the  n-fold  Hilbert  space)  symmetric  tensor 
product  of  H^  with  itself.  On  Hg.  we  define  the  inner 

product  such  that 

where^3”^10'  *  ~K(n)  ’gn] 

^  <fl-afn^lVr(l)fi-ifi(n)-  (3T> 

From  ( 3h )  and  (36),  that  the  mapping 

:F(f,)...F(f  ):  Sym(f..fi. .  .fif  ) 

In  In 

extends  uniquely  to  the  unitary  operator  from  T(h)  onto 

n 

H  .  We  use  this  mapping  to  identify  T(h)  and  H  . 
n  n  n 

Analogous  to  the  fact  that  the  one-dimensional  Hermite 

2  — 1./2  —  2  /p 

polynomials  span  L  (R,3(R)  ,(2ir)_  e-3C  dx) ,  Segal  proved 
r(K)<=oHn«  (38) 


(•see  Miller  [  9  ].  This  derivation  is  somewhat  formal 
because  the  condition  for  the  full  expansion  in  (29) 

to  hold  is  that  E[exp/tb2(s)ds]<“,  which  is  valid 
only  for  small  t. 

U.  Relationship  to  Second  Quantization 
(After  Segal  and  Kelson). 

Let  H  be  a  real  Hilbert  space  and  let  F:H-*-RV(f),A,u) 
be  the  unit  Gaussian  determined  random  field.  If 
f  1 . . , f n  are  orthonormal  in  H  and  4>  is  a  Bounded  Baire 

function  on  Rn,  then 

^(F(f1),..,F(fn)=T|?)n/2/n(fr(x)e-IWI2/2dx 

For  concreteness  (12, A, u)  may  be  chosen  to  be  countably 

infinite  copies  of  (R,B(R),  x  ^2dx). 

If  E  donotes  expectation  on  (ft,A,p)  then 

EfFtfjJ-.Flfg^))^  (32) 

E(F(f. ) . .F(f-  ) )=E[f .  ,  f.  |  —  Iff,  ,  f.  1  (33) 

1  2"  *1  31  3n 

where  the  sum  is  over  all  pairings  of  1,  . . . ,2  ,  i.e. 

il<"  *  ■<in"’  1l<Jl,,'-,in,Jn*  and 

(i. »J, .... ,i_, J_ )  is  a  permutation  1....2  . 

_  •  n  n  n 

Lp(f!,A,W)  is  denoted  by  I.  (H)  and  r0l)  denotes 

L2(H).  r(H)<n  be  the  closed  linear  span  in  T(H) 
of  all  elements  of  the  form  F(f-)...F(f  )  with  m<n 
and  let  r(ll)n  denote  the  orthogonal  complement  of 


for  arbitrary  real  Hilbert  space  H.  T(H)  is  Fock  Space. 

If  the  random  field  F(f)=/fd3,  where 

feL  (R)=H  and  B  is  the  standard  Wiener  process,  the  el¬ 
ements  of  r(H)  are  multiple  Wiener  integrals  (in  the  sense 
of  Ito).  n 

The  space  T ( H )  is  intrinsically  attached  to  the 
structure  of  H  as  a  real  Hilbert  space.  Thus  if  U:H-*-K 
is  an  orthogonal  mapping  of  one  real  Hilbert  space 
into  another,  it  induces  a  unitery  mapping  T(U)  :T(H)-»r(K) , 

where  on  H  ,  r(U)=U3.  ..fflj.  Similarly  if  I:H-»K  is  an 
n  n-foud 

isometric  embedding  then  it  induces  an  isometric  embed¬ 
ding  r(l):  r(HW(K)  and  similarly  for  an  orthogonal 
projection  E : H^K .  If  A : H^K  is  a  contraction  then 
r(A):r(H)-*r(K)  is  defined  to  be  the  direct  sum  to 

r(A)  :H  -+K  ,  where  T(a)  =A3.-.?A.  How  any  contraction 
n  n  n  n  n-fold 

A:H*K  can  be  decomposed  as 

H  -  I-wHHK — H__».kHH 


where  I,  U  and  E  are  as  above. 

Hence  T(A)=r(E)r(U)r(l) .  How  T(A)  is  doubly 
Markovi an  in  the  sense  that 

a>C-»r(A)a>0 

rlA)i=i 

Er(A)a=Ea.  (39) 

Any  doubly  Markovian  operation  is  a  contraction  from 
Lp  to  Lp. 

It  turns  out  thatr(A)  hns  stronger  contractive 
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properties  and  the  precise  statenent  of  this  is  an  im¬ 
portant  theorem  of  Nelson.  Before  we  discuss  this 
result  it  i3  useful  to  recall  that  conditional 
2 

expectations  on  L  (CJ,A,u)  can  be  characterised  as  linear 
positivity  preserving  operators  which  are  idempotant, 
of  norm  £1  and  preserve  constants.  We  also  know  that 
for  p  [l,“],  Pi*2,  all  linear  operators  T  on  LP(52,A,u), 
which  are  idenpotent,  contracting  and  such  that  Tl=l 
is  necessarily  a  conditional  expectation. 

Theorem  3.1  (Nelson  Hvoercontractivity  Theorem) . 

Let  A:H-*K  be  a  contraction.  Then  r(A)  is  a 

contraction  from  L  (H)-*L^(K)  for  l£q£p£”  provided  that 

IWI<(^j)1/2  (*10) 

If  (U0)  does  not  hold  then  l(A)  is  not  a  bounded 
operator  from  Lq(HV>L?(iC) . 
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